Using Mean Shift for Video Image Segmentation

The goal of this project was to develop a fast video image segmentation routine which could be used as a preprocessing step for motion tracking. We chose mean shift [1] as the primary algorithm. Our implementation includes several enhancements including dynamically adjusting the kernel bandwidth based on the overall level of image noise, and keeping a cache of past moves to avoid repeated computations. The frame rate of the resulting implementation is still slow due to the quadratic complextity and iterative nature of the underlying algorithm, but provides a reasonable starting point for refinements. We conclude with a discussion of several limitations of the mean shift as a procedure for image segmentation and present a couple options for improvement. GENERAL PROBLEM FORMULATION Image segmentation is an important low-level task in computer vision. As we saw in the homework the k-means algorithm can be used to compress an image into a reduced color space that approximately represents the original image. A robust segmentation could be the first stage in a object tracking pipeline, and in some ways represents a dual approach to the more common approach of tracking corner features. Informally, we define the video image segmentation problem as follows.